The automatic classification of document collections into categories is an increasingly important task. Examples of document collections that are often organized into categories include web pages, patents, news articles, email, research papers, and various knowledge bases. As document collections continue to grow at remarkable rates, the task of classifying the documents by hand can become unmanageable. However, without the organization provided by a classification system, the collection as a whole is nearly impossible to comprehend and specific documents are difficult to locate.
The present invention offers a unique document processing approach. In accordance with the teachings of the present invention, a computer-implemented system and method are provided for processing text-based documents. A frequency of terms data set is generated for the terms appearing in the documents. Singular value decomposition is performed upon the frequency of terms data set in order to form projections of the terms and documents into a reduced dimensional subspace. The projections are normalized, and the normalized projections are used to analyze the documents.